BMC Medical Informatics and Decision Making

Objective: Retrospective evaluation of clinical pathway adherence using electronic medical record (eMR) data is limited by heterogeneous data structures and statistical measures that do not account for patient-specific context, care sequence, or timing. We designed a modular event-based architecture to dynamically evaluate the New South Wales Adult Sepsis Pathway over time. Method: Data from eMRs were collected from adult emergency department encounters across four hospitals between July 2022 and June 2024 and processed through three sequential layers: event-stream transformation, pathway-relevance filtering, and state-based pathway evaluation. Heterogeneous records were standardized and organized into a chronological timeline for each encounter. The framework then updated the pathway state as new clinical information became available and assessed whether recommended care was delivered within the relevant timeframe and sequence. Result: The architecture processed 451,124 encounters. This included 57,631,084 clinical events extracted and transformed into encounter-level event streams. After filtering for pathway relevance, 43,951,146 events remained (76.3%). A total of 129,907 (34.6%) encounters entered the sepsis pathway, 50,118 (38.6%) entered YELLOW state, indicating progression to suspected sepsis, and 3,349 (2.6%) encounters progressed to RED state from GREEN, indicating the initial shock. Of 50,118 encounters entering YELLOW state, 2,426 (4.8%) progressed to shock, 8,700 (17.4%) encounters were recognized with sepsis through a blood culture order, and 4,287 (8.6%) encounters were recognized with sepsis through an antibiotic order. This demonstrates the ability of the architecture to prioritize higher-acuity states during evaluation rather than going through an intermediate state. Conclusion: The architecture developed shows the feasibility of transforming fragmented routine eMR data into a reusable longitudinal representation that supports context-dependent pathway evaluation. Separating data preparation from pathway rules improves transparency, maintainability, and adaptability and provides a foundation for future interoperable, near-real-time pathway monitoring

Background: As required by the 21st Century Cures Act, all test results, including final radiology reports, are released to patients as soon as the results are finalized. However, radiology reports are primarily intended for communication between medical professionals and can often be difficult for patients to interpret, particularly in cancer care settings. We developed a tool that leverages a large language model (LLM; ChatGPT-5.2, OpenAI, San Francisco, CA) to draft a patient-friendly plain language version of imaging results (Patient-Friendly Scan Result Generation [PFSG] tool) that oncologists can incorporate into their communication with patients via patient portals. This article describes our mixed-methods study protocol. Objective: Our primary aims are to: 1) assess whether patients who receive patient-friendly versions of scan results report better understanding of their results than patients who do not; 2) assess whether introduction of the PFSG tool affects provider workload; and 3) explain how and under what circumstances the PFSG tool enhances communication and workflows, and affects patient and provider outcomes. Methods: We will use an embedded mixed-methods, prospective cohort design within a quality improvement (QI) initiative. The PFSG tool will be implemented using a modified three-wave staircase trial design, with providers in randomized cohorts of oncology disease groups progressively gaining access to the tool over the course of the study. Approximately 60 providers from 11 oncology disease focus groups will be randomized. Longitudinal provider outcomes will be measured through surveys at one month pre-implementation, one month post-implementation, and three months post-implementation. Patient outcomes will be measured cross-sectionally through surveys at one month pre-implementation (n=350) and at post-implementation result point-of-contact (n=350). Additionally, we will conduct interviews with patients and oncologists to inform the iterative design of the tool and its implementation. Results: This study was funded in September 2025 and data collection began in February 2026. As of September 2026, the PFSG tool has been rolled out to all oncology providers, with Cohort 3 post-implementation data collection beginning in August 2026. A total of 738 patient baseline surveys have been collected and patient post-implementation data collection is in progress. Conclusions: This is the protocol for a study of the implementation of a patient-friendly results tool in oncology provider workflows to facilitate improved communication of radiology results to oncology patients. The results of this study will lend evidence to the utility and acceptability of the study tool, and tools like it, to close an important oncology care gap. If the tool is shown to improve key oncology patient and provider outcomes, it may lead to development of a wider release for more general patient populations.

Accurate prediction of adverse events in the intensive care unit (ICU) following cardiothoracic surgery is crucial for timely interventions, improving patient outcomes and optimizing value in healthcare. By leveraging advanced machine learning techniques, this study demonstrates the transformative potential of predictive analytics to enhance postoperative care using a dataset from the Society of Thoracic Surgeons' (STS) database and time-series intraoperative data. We developed a multi-modal late fusion approach integrating static patient variables with intraoperative time-series data, utilizing an ensemble of models including residual neural networks for static features, CNN and Bidirectional GRU for time-series processing, XGBoost, logistic regression, SVM and Gradient Boosting. Through five-fold cross-validation, our ensemble model with balanced threshold selection achieved an AUC of 0.87, sensitivity of 0.76, specificity of 0.83, PPV of 0.40, and NPV of 0.96. Preoperative heart failure, IABP insertion, cardiopulmonary bypass (CPB) duration and white blood cell (WBC) count emerged as key predictors. Training the same pipeline on each modality separately showed that the static registry variables alone reached the same AUC of 0.87, and that the intraoperative signals alone reached 0.83, so on this cohort the two modalities act as substitutes rather than complements and fusion changed the sensitivity and precision trade-off rather than the amount of separable signal. A per-signal ablation found that no individual monitoring channel was load bearing, with mean arterial pressure and central venous pressure the least substitutable. This balanced approach optimizes the trade-off between sensitivity and specificity, providing a screening tool with high negative predictive value for monitoring high-risk patients, and the modality comparison indicates that comparable performance is attainable at sites holding registry data alone.

Background: Patients with intracerebral hemorrhage (ICH) in intensive care units (ICUs) frequently receive antibiotics early in critical illness. Such exposure reflects suspected infection, physiological deterioration, monitoring intensity, and clinical management and may vary across institutions and electronic health record databases. We evaluated target-domain adaptation for modeling sustained antibiotic exposure in critically ill patients with ICH. Methods: We analyzed derived ICH cohorts from MIMIC-IV and the eICU Collaborative Research Database. MIMIC-IV included 2,024 ICU stays and served as the source cohort. The eICU cohort comprised 1,277 stays and was divided, at the ICU-stay level, into a development subset (766 stays) and a held-out evaluation subset (511 stays). The eICU outcome was sustained antibiotic exposure on at least two ICU days. Candidate models incorporated demographic, comorbidity, physiological, laboratory, neurologic, respiratory, temporal, and measurement-process features. Timestamped observations for static and temporal tabular families were censored at the recorded first-antibiotic time when available, whereas the compact sequence component and explicit measurement-density features used the full first 24 hours. For base-model families using eICU development data, meta-learner inputs were generated by five-fold stratified cross-fitting. A 42-input L2-penalized logistic meta-learner combined 16 model probabilities with model-summary, clinical, availability, and measurement-density features. Results: The held-out eICU subset contained 48 positive outcomes (9.39%). The adapted ensemble achieved an area under the receiver operating characteristic curve (AUROC) of 0.841 (95% CI 0.775-0.899) and an average precision (AP) of 0.540 (95% CI 0.399-0.676), with a Brier score of 0.063 and expected calibration error of 0.029. The MIMIC-IV source-only stack achieved an AUROC of 0.782; the paired AUROC difference was 0.059 (95% CI 0.003-0.112). In a separate patient-grouped five-fold out-of-fold sensitivity analysis of 503 linked eICU stays, AUROC was 0.807 (95% CI 0.691-0.910). Conclusions: Target-domain adaptation improved held-out discrimination relative to a source-only stack for sustained antibiotic exposure. Because some model components and measurement-process features used information from the broader first 24 hours, the model should be interpreted as a first-24-hour treatment-association model rather than a strictly prospective pre-treatment infection predictor.

De-identifying temporal protected health information (PHI) is challenging because temporal expressions appear in diverse formats and must be transformed while preserving clinically meaningful timelines. This study proposes and evaluates an LLM-based framework for temporal PHI de-identification in real-world clinical notes. Using 1,148 notes from 30 sarcoma patients (28,431 annotated temporal entities), we evaluated multiple modern LLMs for temporal entity extraction and surrogate generation. Proprietary models achieved strong extraction performance, with GPT-4o achieving the best overall results. Most models preserved temporal formatting (>99%), but surrogate generation remained challenging. GPT-5.4 achieved the highest shift correctness (90.2% on a shared-entity subset) and the lowest order violations. Error analysis revealed systematic shift deviations ({+/-}1, {+/-}30/31, {+/-}365 days), highlighting persistent limitations in LLM temporal reasoning. These findings suggest that reliable temporal de-identification will require hybrid or multi-agent approaches beyond standalone LLMs.

Background: Adjuvant therapy decision in early-stage node-negative breast cancer remains one of the most challenging problems in oncology. Although most patients in this population achieve favorable outcomes with endocrine therapy alone, a subset remains at risk of relapse and may benefit from chemotherapy. Accurate identification of high-risk and low-risk patients is therefore essential to prevent both undertreatment and overtreatment. Methods: In this study, machine learning models were assessed using MSK-IMPACT (n=645) and METABRIC (n=503) cohorts comprising node-negative stage I-II breast cancer patients. Three experimental settings were evaluated: training on MSK-IMPACT and testing on METABRIC, the reverse direction, and five-fold cross-validation on the combined cohort. Sensitivity analysis and data harmonization were applied to identify and mitigate cohort-specific bias. Survival analysis, SHapley Additive exPlanations (SHAP) interpretability, and ablation analyses were performed to validate clinical relevance and characterize feature contributions. Results: The combined cohort model achieved an area under the receiver operating characteristic curve (AUC) of 0.788{+/-}0.028 across five-fold cross-validation, while bidirectional cross-cohort validation yielded AUC of 0.741 when testing on MSK-IMPACT and AUC of 0.714 when testing on METABRIC. Survival validation on METABRIC out-of-fold predictions from the combined cohort model showed hazard ratios of 3.01 (95\% CI: 2.28-3.98) for disease-free survival and 2.35 (95\% CI: 1.79-3.09) for overall survival, consistent across Luminal A, Luminal B, and Triple-Negative subtypes. Conclusion: Validated across two geographically independent cohorts, the proposed model enables cost-effective, gene expression-free identification of patients at high-risk of distant relapse and may facilitate adjuvant chemotherapy decision-making in early-stage node-negative breast cancer.

Background: Conventional discrimination metrics, such as the area under the receiver operating characteristic curve (AUROC), do not characterize the composition of outcomes among patients flagged as high risk by clinical prediction models. This limitation is particularly relevant in critical care, where competing deaths and incomplete follow-up can complicate the interpretation of prediction alerts. We developed the Balanced Risk Integrity Index (BRII), a stratum-level metric defined as the proportion of target events among target events, competing deaths, and early-censored observations in a flagged high-risk stratum. BRII is algebraically related to positive predictive value (PPV) and is reported alongside its component decomposition to characterize the outcome composition of flagged patients. Methods: We evaluated BRII using seven acute kidney injury (AKI) prediction models developed in MIMIC-IV and externally validated in MIMIC-III and eICU-CRD (116,010 patients in total). Additional simulations examined the effects of competing deaths and early censoring on BRII and conventional model performance metrics. Results: For 48-hour AKI prediction in the top 10% risk stratum, BRII ranged from 0.90 to 0.96 in MIMIC-III, compared with less than 0.26 across all models in eICU-CRD, where 61.5% of evaluated patients were censored before the prediction horizon. In the eICU-CRD top 10% strata for the same target, 37.0% to 50.2% of patients were censored early, 12.2% to 15.5% had AKI, and 3.4% to 8.1% died before AKI. BRII and PPV ordered at least one model pair differently in 34 of 48 cohort-outcome-top-k groups. Under simulated random censoring for 48-hour AKI prediction, BRII declined by up to 0.80 while binary-label AUROC changed by less than 0.02. Conclusions: BRII provided complementary information on high-risk stratum composition, yielding different model rankings from PPV and revealing substantial differences across external validation cohorts. Interpreted alongside PPV and its component decomposition, BRII offers a standardized approach to evaluating clinical prediction alerts across models and observation environments.

Background: Intracerebral hemorrhage (ICH) is associated with high early mortality. Although routine coagulation markers are clinically relevant, their incremental value for mortality prediction beyond neurologic and physiologic information is uncertain. We developed and externally validated models using first-24-hour ICU data and evaluated the added predictive value of routine coagulation measurements. Methods: We used MIMIC-IV for model development and eICU-CRD for external validation. The development cohort included 2,669 adults with non-traumatic ICH and 388 ICU deaths. Logistic regression, random forest, and XGBoost were evaluated using a patient-level 70:30 holdout split and 5-fold stratified cross-validation. External validation used a high-specificity primary eICU cohort (396 patients; 39 deaths) and a broader secondary hemorrhagic-stroke/likely-ICH cohort (1,555 patients; 168 deaths). Performance was assessed using AUROC, AUPRC, Brier score, calibration, and paired bootstrap comparisons. GCS-only and age-plus-GCS models served as clinical benchmarks. Coagulation-only, clinical-only, and combined XGBoost models were compared to assess incremental value. Results: In the internal holdout cohort, AUROCs were 0.870 for logistic regression, 0.859 for random forest, and 0.867 for XGBoost. In the primary external cohort, corresponding AUROCs were 0.804, 0.839, and 0.817; AUPRCs were 0.342, 0.419, and 0.406; and Brier scores were 0.080, 0.072, and 0.074. In the broader cohort, AUROCs were 0.826, 0.848, and 0.838. Random forest showed the numerically strongest external performance, but pairwise AUROC differences among multivariable models were not clearly supported. GCS-only achieved AUROCs of 0.780 and 0.813 in the primary and broader cohorts. Coagulation-only XGBoost showed weak discrimination (AUROC 0.598 internally, 0.598 primary external, and 0.566 broader external). Adding coagulation variables to the clinical model produced AUROC differences of 0.014 (95% CI -0.009 to 0.037) and 0.003 (95% CI -0.007 to 0.013) in the two external cohorts. Conclusions: Multivariable models based on first-24-hour ICU data retained useful discrimination across external cohorts and provided modest gains over simple GCS-based benchmarks. Routine coagulation markers alone were weak predictors and did not provide clearly supported incremental discrimination beyond broader clinical information. Cross-database transportability should be interpreted together with phenotype alignment and calibration.

Background and Objective: The volume of patient portal messages has grown substantially and artificial intelligence (AI) can support clinical staff and improve the timeliness of patient care. Our objective is to test if an AI tool for managing the electronic inbox In-Basket reduces response times to patient portal messages in neurology. Methods: We conducted an implementation study from January to May 2026 in the neurology department at two clinical sites within a regional medical system. A single AI tool was implemented to auto-categorize and auto-route all patient messages received in the electronic inbox. The tool also generates optional draft replies to messages. Primary process measures were the median weekly time (hours) from message creation to the first action and to closure. Impact was analyzed using a controlled interrupted time-series (ITS) design, comparing message-volume-matched control divisions. Results: Among 24,142 messages analyzed, pre-intervention median weekly time to first action was 11.3 hours [IQR: 5.6, 15.1] (Site A) and 16.1 hours [IQR: 13.8, 19.0] (Site B), falling post-intervention to 1.0 hour [IQR: 0, 3.1] (Site A) and 0.6 hours [IQR: 0.2, 2.4] (Site B). Controlled interrupted time-series analysis showed a significant immediate differential level change in median weekly hours to first action at Site A (-92.5%, 95% CI: -97.5% to -78.0%), and a significant downward trend at Site B (-43.5%, 95% CI: -65.2% to -8.2%). Changes in time to closure were directionally favorable but not statistically significant at either site. There was low use of the generated draft replies, only 0.5% at Site A and 3% at Site B. Discussion: Implementation of AI-assisted message categorization and routing was associated with a substantially reduced initial action time for patient In-Basket messages in Epilepsy. The utilization of AI-generated draft-replies to messages was low. These findings support the feasibility and effectiveness of AI-augmented workflows to improve inbox management.

Electronic health records provide a source of real-world information for disease risk prediction, but many machine learning models learn implicit associations that are difficult to inspect or use for clinical reasoning. Causal Bayesian networks (CBNs) offer a form of causal-interpretable machine learning by representing conditional dependencies, putative directional relations, and probabilistic evidence propagation in a directed acyclic graph. However, conventional CBN structure learning becomes unstable and computationally expensive when applied to large-scale EHR data. To address this limitation, we propose UPEBNL, a scalable framework for causal-interpretable cancer prescreening based on parallel CBN learning. UPEBNL integrates adaptive data slicing, quality-aware structure aggregation, and global DAG construction to learn stable and interpretable dependency structures from large-scale observational data. We evaluated UPEBNL in high-dimensional and multi-million-sample simulations and applied it to EHR-based prescreening for esophageal and colorectal cancer. In simulations, UPEBNL improved structural recovery accuracy by nearly 40% and achieved up to a 221.28-fold speedup over conventional CBN learning strategies. For cancer risk prediction, the learned CBNs provided interpretable evidence paths and achieved validation AUCs of 0.8171 for esophageal cancer and 0.784 for colorectal cancer. Calibration and decision curve analyses further supported the reliability and clinical utility of the models. These findings suggest that scalable CBN learning can support interpretable cancer prescreening from large-scale EHR data.

Biomedical artificial intelligence (AI) increasingly relies on heterogeneous data sources, yet differences in data structure, terminology, evidence quality, and provenance complicate their integration into reliable knowledge resources. Here, we develop an evidence-aware framework for processing and integrating heterogeneous pharmacovigilance data from FDA Adverse Event Reporting System (FAERS) reports, biomedical literature, clinical trials, and electronic health records in Mass General Brigham. The framework combines large language model (LLM)-based information extraction, ontology-driven semantic normalization, cross-source evidence assessment, statistical and machine learning-based prioritization, and ingredient-based adverse drug event (ADE) relationship inference to transform heterogeneous evidence into a structured and evidence-based knowledge resource. The constructed knowledge system contains 11,588 drug nodes, 44,843 ingredient nodes, 3,275 ADE nodes, 993 indication nodes, and more than 2.47 million relationships, with each drug-ADE relationship assigned a confidence score. We further developed a drug-level web interface for accessing the knowledge resource and validated its utility in downstream AI tasks. Specifically, incorporating the developed knowledge resource improved the comprehensiveness, relevance, and overall quality of LLM-based drug-specific ADE synthesis and improved the precision and F1 score of clinical ADE relation extraction. These results demonstrate that AI-assisted processing combined with evidence-aware curation can transform heterogeneous biomedical evidence into reusable, AI-ready knowledge with explicit provenance and evidence strength information. The framework provides a general strategy for integrating heterogeneous biomedical data to support reproducible knowledge construction and downstream biomedical AI applications.

In hypertension screening, previously diagnosed cases provide reliable positive labels, whereas individuals without a prior diagnosis comprise both undiagnosed hypertensive and non-hypertensive individuals, creating a canonical positive-unlabeled (PU) learning setting. We propose PU-Boost, a two-stage reliable-sample reconstruction framework that uses only readily obtainable demographic, behavioral, medical-history, and anthropometric variables. During training, only previously diagnosed hypertension cases are treated as labeled positives (P), while all remaining individuals are treated as unlabeled (U); no additional reference labels are supplied to the two-stage reconstruction procedure. Elastic Net first provides a global ranking of P versus U, after which a random forest refines the uncertain region and highly ambiguous observations are rejected. XGBoost is then trained on the reconstructed class-balanced pseudo-labeled sample. In the independent test set, PU-Boost achieved a sensitivity of 0.817 and a balanced accuracy of 0.682, the highest observed values among the compared models. It produced 729 false negatives, 41.1% fewer than standard XGBoost, although specificity decreased to 0.547. SHAP analysis indicated that established hypertension-related features, including age, body mass index, waist circumference, and family history, contributed strongly to model output. PU-Boost therefore provides a statistically motivated framework for reducing missed cases when screening for undiagnosed hypertension under incomplete diagnostic labeling.

Background: Machine Learning (ML) has shown promise in histopathological diagnosis, but rare tumours, like duodenal epithelial neoplasms, are challenging because large, labelled datasets are difficult to assemble. We propose a data-efficient ML pipeline for detecting duodenal epithelial neoplasia in whole-slide images. Methods: We assembled a five-hospital dataset of haematoxylin and eosin-stained duodenal biopsy whole slide images (WSIs) at 10x magnification, comprising 8,027 normal and 401 neoplastic biopsies, including adenomas, carcinomas and neuroendocrine tumours. WSIs were divided into patches and encoded using the Hibou-B foundation model. We compared global standardisation with unsupervised site-specific standardisation of patch embeddings. Our pipeline combined principal component analysis, a Gaussian mixture model and a Random Forest classifier. We assessed generalisability using leave-one-hospital-out cross-validation across four hospitals and on an independent test set from a fifth hospital. Results: In leave-one-hospital-out cross-validation, the pipeline achieved a mean accuracy of 90%, sensitivity of 96%, specificity of 83% and an AUC of 0.96. On the external test hospital, it achieved 94% accuracy, 93% sensitivity, 95% specificity, and AUC of 0.97. Site-specific standardisation improved mean cross-validation accuracy by 20 percentage points compared with global standardisation, while the Gaussian mixture pipeline improved accuracy by 12 percentage points compared with mean pooling of patch embeddings. Conclusions: We present a data-efficient ML approach for detecting duodenal epithelial neoplasia from WSIs. Strong performance was maintained across hospitals and on an independent external test cohort despite the limited number of neoplastic training cases, supporting the feasibility of foundation-model-based approaches for rare histopathological diagnoses.

Background: Digital systems are central to regulating access to care in Brazil's Unified Health System (SUS), but fragmented data can hinder coordination. Objective: This study examines the database structure of SUSfacilMG and identifies modernization priorities, using CORE Saude MG as a comparative reference. Methods: We assessed metadata from 1,586 SUSfacilMG tables, focusing on naming patterns, relationships, and repeated attributes. We then compared architectural features described for CORE Saude MG. Results: Approximately 40% of the assessed tables repeated descriptive attributes, including patient identifiers, suggesting substantial duplication. Metadata alone cannot establish operational failures, security incidents, or effects on care. Conclusion: A phased review of data models, interfaces, and governance is warranted. Standardized exchange, referential integrity, and controlled access should be evaluated alongside service continuity and data quality.

Tabular synthetic data generation (SDG) can facilitate research and development in the health domain, as access to health data is restricted. To accommodate for its structural complexity, such as multimodal distributions and fragmentation, transformer models provides an alternative to conventional SDG methods since they are better equipped to infer global spatial coherence due to textual context of tabular records. However, methods for optimising their adaption to tabular SDG are still largely unexplored. We contribute to the understanding of transformer-based SDG by evaluating fidelity and privacy of three prompting regimes, zero shot, one shot and few shot training, for tabular health SDG using small transformer models. We evaluated llama 3.2 (1B and 3B parameters) and Phi (1.3B and 2.7B parameters), on a real clinical dataset. Our results showed that zero shot learning was less prone to hallucinations and performed overall better for fidelity, but at the loss of privacy in comparison to the other regimes. Our results showed no differences between model sizes and families. Future studies should extend the analysis to other models and datasets, and explore strategies to reduce hallucinations.

Objective: To determine how cohort versus nested case-control (NCC) design affects EHR-based prediction of a rare outcome. Materials and Methods: We compared two cohort and five NCC designs for predicting early-onset gastrointestinal cancer using L1-regularized logistic regression (LASSO), XGBoost, and multilayer perceptron (MLP). Models underwent nested five-fold cross-validation with model-specific hyperparameter tuning and temporal evaluation in a held-out prospective cohort representing intended deployment. Model discrimination was assessed using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). Model calibration was assessed using Brier scores and calibration plots before and after recalibration with isotonic regression. Feature-ranking similarity was quantified using rank-biased overlap. Results: The cohort development datasets included 256,854-415,529 patients, with 2-yr cumulative incidence of 0.12%-0.17%, compared with approximately 9% in the 1:10 matched NCC datasets. NCC designs generally produced higher internal discrimination, but temporal performance varied by algorithm and index-date definition. Cohort designs achieved the highest temporal AUPRC for LASSO (0.068), whereas diagnosis-anchored NCC achieved the highest temporal AUPRC for XGBoost (0.086) and MLP (0.070). Temporal AUPRC generally decreased as the required lead time before diagnosis increased. Raw predicted probabilities substantially overestimated observed risks in the temporal cohort. Isotonic recalibration improved probability scaling while largely preserving discrimination, although residual miscalibration remained. Cohort-based feature rankings were more consistent across algorithms, whereas NCC-based rankings depended more strongly on algorithm and index-date anchoring. Conclusion: Study design should be selected and evaluated according to the intended clinical prediction task and deployment setting.

Objective: To develop and evaluate a framework for characterizing clinician editing of Artificial Intelligence (AI)-generated[KS1.1] documentation and assess its feasibility for health system-level monitoring of AI scribes. Materials and Methods: We analyzed outpatient encounters in which an AI scribe was used at a single academic health system, examining the History of Present Illness (HPI) and Assessment and Plan (A&P) sections of notes. We characterized edits on three dimensions[KS2.1][AG2.2]: lexical edit intensity based on Levenshtein distance, embedding edit intensity BERT[KS3.1]Score, and clinical edit intensity based on removed and added UMLS[KS4.1][AG4.2] concepts.[KS5.1][AG5.2] We analyzed the Positive Predictive Value (PPV) of clinical edit intensity as a measure of clinically meaningful editing using clinicians as the gold standard, examined correlations among dimensions, and designed exponentially weighted moving-average control charts to monitor longitudinal changes in clinician editing behavior.[KS6.1][AG6.2] Results: 268,379 encounters were included (267,654 HPI, 267,594 A&P). Clinical edit intensity [≥]1 had an 88.9% PPV for clinically meaningful editing. Lexical and embedding edit intensity were highly correlated (Spearman {rho} 0.95), while clinical edit intensity was less strongly correlated with both ({rho} 0.77-0.81). Longitudinal monitoring detected changes coinciding with system-wide rollout.[KS7.1][AG7.2] Discussion:[KS8.1][AG8.2] Clinicians edited A&Ps more heavily than HPIs, potentially reflecting greater attention to content involving clinical decision-making. Over one-third of sections involved clinical concept changes, and clinical edit intensity identified clinically meaningful edits while providing information complementary to lexical editing measures. Conclusion: Clinician editing can be characterized at scale using complementary editing dimensions, providing a scalable signal for post-deployment surveillance of the human-AI documentation process.

Metabolic dysfunction-associated steatotic liver disease (MASLD) is highly prevalent among individuals with type 2 diabetes (T2D), yet accurate identification from electronic health records (EHRs) remains challenging because clinically relevant information is fragmented across structured data and unstructured clinical documentation. We developed a rule-based computable phenotype (CP) framework that integrates structured EHRs with clinical notes for scalable MASLD identification in patients with T2D. The framework comprises five complementary CP algorithms derived from validated fibrosis indices and liver injury biomarkers, together with a lightweight natural language processing module that identifies MASLD-related evidence using keyword matching, contextual filtering, and negation detection. The proposed framework was systematically evaluated across multiple threshold settings, with additional validation through age- and sex-stratified analyses and manual chart review. The findings demonstrate that integrating structured and unstructured EHR data provides an accurate and scalable framework for real-world MASLD phenotyping and offers a practical strategy for developing CPs for other chronic diseases.

Decision models like Jev read text and return probabilities, categories, or ordinal levels, with confidence scores for the latter two, at a fraction of large language models' (LLMs) cost. We explored how Jev could be integrated into clinical workflows for all these output types: mortality risk estimation, categorical Choosing-Wisely decisions for patient questions, and ordinal colon cancer staging. With standard LLM-style inputs, Jev trailed five flagship LLMs: mortality AUROC was 0.822 versus 0.837-0.846; Choosing-Wisely accuracy, 87.7% versus 98.4-100%; and staging accuracy, 45.4% versus 94.2-100%. Developer-recommended input alterations improved staging accuracy to 75.8% with medical context, 82.1% with task decomposition, and 85.8% with both. None of Jev's staging errors had full confidence. Using Jev for fully confident staging decisions and LLMs for the rest was non-inferior to each LLM alone and cut flagship LLM costs by 33-44%. Prompted as designed, decision models may efficiently share structured clinical tasks with LLMs.

BackgroundEvaluating fairness in artificial intelligence (AI) is essential before clinical implementation to prevent unintended harm across diverse patient populations. This study assesses performance differences of an image-based surgical site infection (SSI) detection model across age, gender, geography, race, and skin color. MethodsModel development/validation study using retrospective, multicenter data across surgical subspecialties from adults who submitted an image of their incision via the electronic health record within 30 days of their procedure between 2019-2024. Area under receiver operating curve (AUROC) was compared by age (<55 vs. [&ge;]55 years old), gender (male vs. female), geography (Rochester, Minnesota vs. Jacksonville, Florida vs. Scottsdale, Arizona vs. regional Minnesota/Wisconsin areas), race (White/Caucasian vs. non-White/Caucasian), and skin color (Fitpatrick I-III vs. IV-VI and Monk A-E vs. F-J). ResultsTraining data contained 13,702 images from 4,274 patients between 2019-2022 (50.1% [&ge;]55 years old; 63.2% female; 91.6% White/Caucasian; 58.5% Rochester, 17.3% Florida, 10.5% Arizona; 96.3% light skin tone), of which 1,348 images (9.8%) had an SSI. No differences in SSI detection existed for age, gender, race, or skin color (AUROC 0.83-0.87, p>0.09), however geographical differences were significant: lowest AUROC 0.83 (95% CI 0.77-0.88) vs. highest AUROC 0.87 (95% CI 0.84-0.89, p=0.01). Temporal validation using 4,242 images from 1,353 patients between 2022-2023 showed intact fairness across subgroups ({Delta}AUROC [&le;]0.06) except geography ({Delta}AUROC [&le;]0.16). ConclusionsThe model behaved fairly across age, gender, race, and skin color, however geographical differences persisted. These findings have practical implications underscoring the importance of evaluating bias in clinical AI before deployment.